IREX 10: Demographics

The figure below shows the FNIR at FPIR=0.01 (t = 2700) for different demographic groups. The bars show 95% confidence intervals.

Dataset:	Operational Dataset 4th pull
Samples used:	Both eyes
Enrolled Population:	500K people
Enrollment Method:	One enrollment session per person

Some consolidation of demographic information was necessary to improve statistical power. Eye color was consolidated to either light (grey, blue, or green) or dark (brown or black). Some subjects were labeled as being neither male nor female. Meaningful results for these categories could not be obtained because their sample sizes are too small. For the same reason, results for races other than white and black are not shown. The precise definitions of race, sex, and eye color used here can be found in EBTS version 10.0.

Logistic Regression

This section models the relationship between FNIR and various demographic characteristics using logistic regression. The response variable is whether the search produces a false negative at an FPIR of 0.01. The precise logit relationship is

$\ell =\log \frac{p}{1-p}={\beta }_0+{\beta }_1\mathbf{Sex}+{\beta }_2\mathbf{Race}+{\beta }_3\mathbf{Eye\; color}+{\beta }_4\mathbf{Sex\; and\; Eye\; Color}+{\beta }_5\mathbf{Sex\; and\; Race}$

where p is the probability of a false negative and ℓ is the log likelihood ratio of the probability of a false negative.

McFadden’s $R^2$ = 0.0000438

n = 311,452

Negative (blue) values mean the probability of a miss is decreased. McFadden’s pseudo $R^2$ is a measure of the goodness-of-fit that produces values between 0 and 1. Race, sex, and eye color are generally poor predictors of accuracy, so the value is typically low.

The model does not include any interactions between race and eye color because there were not enough cases of blacks with light eyes to produce meaningful results. Eye color was unavailable for some subjects so MICE was used to perform imputation.

Demographic breakdown of false positives

The figures below show the demographic composition of false positives. Most false positives are likely to be white and male since they comprise the majority of the test data. The precise demographic breakdown of the test data is

• 68.5% White and male
• 16.8% White and female
• 12.8% Black and male
• 1.9% Black and female

and

• 85.5% Dark eyes
• 14.5% Light eyes

Other races, sexes, and eye colors are ignored due to the infrequency of their occurence in the test dataset.

Below each figure is a calculation of demographic fairness, the geomtric mean from FRVT: Summarizing Demographic Differentials. The calculations are not exact matches for Equations 10 and 11 in the report because the one-to-one error metrics FMR and FNMR are replaced with the one-to-many error metrics FPIR and FNIR respectively.

Demographic breakdown of hits

NOTE: Plots may not render if the matcher produces highly discretized scores.